* Import sector-level price rigidity from Michael Weber
* ===========================================================

foreach k in 2 3 4 5 {  

import delimited using "${dirInput}/FrequencyNaics`k'dnew", clear

rename freq_id freq 

* Convert to quarterly
replace freq = 1-(1-freq)^3

* Compute duration
gen dur = -1/(log(1-freq))

* rename naics to match compustat dataset
rename naics`k'd naicsq`k'
rename freq freq`k'd
rename dur dur`k'd

save ${dirTemp}/stickiness_naics`k', replace
	
}

* Create/impute firm-level price rigidity
* ==============================================

import delimited using ${dirInput}/compsegd_wrds_segmerged_ALL, clear

* drop unneeded variables
drop *_fn *_dc 

* drop unneeded sectors
cap drop naics2
gen naics2=floor(naics/10000)
drop if naics2==52 | naics2==53 

* dates
gen aux=string(datadate,"%10.0g")
gen date=date(aux,"YMD")
format date %td
drop aux
gen year=yofd(date)
gen month=month(date)
gen day=day(date)

* drop geo and state segment files
drop if stype=="GEOSEG"
drop if stype=="STSEG"

* drop missing, negative, or zero sales
drop if sales<=0 | sales==.

* BUS and OP sometimes concur
gen aux1=1 if stype=="BUSSEG"
gen aux2=1 if stype=="OPSEG"
bysort gvkey date: egen aux11=mean(aux1)
bysort gvkey date: egen aux22=mean(aux2)
gen aux33=aux11+aux22
sort gvkey date stype 

* drop BUSSEG if OPSEG available
drop if aux1==1 & aux22==1
drop aux* 

* drop duplicates: same gvkey/date/sid/sales/naicss1
bysort gvkey date sid sales naicss1: gen aux=_N
drop if aux>=2
drop aux*

* drop duplicates: same gvkey/date/sid/naicss1, different sales and different source date
bysort gvkey date sid naicss1: gen aux=_N
bysort gvkey date sid naicss1: egen long aux2=max(srcdate)
gen aux3=aux2==srcdate
drop if aux>1 & aux3==0
drop aux*
bysort gvkey date sid naicss1: gen aux=_N 
drop aux*

* drop duplicates: same gvkey/date/sid, different naicss1 and different source date
bysort gvkey date sid: gen aux=_N 
bysort gvkey date sid: egen long aux2=max(srcdate)
gen aux3=aux2==srcdate
drop if aux>1 & aux3==0
drop aux*
bysort gvkey date sid: gen aux=_N
drop aux*

* drop duplicates: same gvkey/sid/year, different months
bysort gvkey year sid: gen aux=_N
bysort gvkey year sid: egen aux2=max(date)
gen aux3=aux2==date
drop if aux>1 & aux3==0
drop aux*
bysort gvkey year sid: gen aux=_N 
drop aux*

* drop firm-year if naicss1 missing for all firm-segments
gen aux=1 if naicss1==.
bysort gvkey date: egen aux2=sum(aux)
bysort gvkey date: gen aux3=_N
gen aux4=aux2/aux3
sum aux4,d
sum aux4 if aux4>0 & aux4<1,d
drop if aux4==1
drop aux*

* replace naicss1 by naics if some firm-segment naicss1 missing
replace naicss1=naics if naicss1==.

* collapse to gvkey/date/naicss1
collapse (sum) sales, by(gvkey year naicss1)
bysort gvkey year: egen sales_sum=sum(sales)
gen sales_share=sales/sales_sum
sum sales_share,d

* Correct naics codes
cap drop naicsq*
replace naicss1=floor(naicss1/10) if naicss1>100000 
gen naicsq5=naicss1 if naicss1>10000
gen naicsq4=naicss1 if naicss1<10000 & naicss1>1000 
replace naicsq4=floor(naicsq5/10) if naicsq5!=.
gen naicsq3=naicss1 if naicss1<1000 & naicss1>100
replace naicsq3=floor(naicsq4/10) if naicsq4!=.
gen naicsq2=naicss1 if naicss1<100 & naicss1>10 
replace naicsq2=floor(naicsq3/10) if naicsq3!=.

* Merge with price adjustment frequency
cap drop freq* dur*
merge m:1 naicsq5 using ${dirTemp}/stickiness_naics5
drop if _merge==2
drop _merge
merge m:1 naicsq4 using ${dirTemp}/stickiness_naics4
drop if _merge==2
drop _merge
merge m:1 naicsq3 using ${dirTemp}/stickiness_naics3
drop if _merge==2
drop _merge
merge m:1 naicsq2 using ${dirTemp}/stickiness_naics2
drop if _merge==2
drop _merge
 
* Start computing weighted average duration/frequency
gen freq=freq2d
replace freq=freq3d if freq3d!=.
replace freq=freq4d if freq4d!=.
replace freq=freq5d if freq5d!=.
gen dur=dur2d
replace dur=dur3d if dur3d!=.
replace dur=dur4d if dur4d!=.
replace dur=dur5d if dur5d!=.
drop if freq==.

* Flag firm-year when rigidity variation comes from max. 4/3/2 digit level
gen flagd=.
replace flagd=5 if freq5d!=. 
replace flagd=4 if freq4d!=. & freq5d==. 
replace flagd=3 if freq3d!=. & freq4d==. 
replace flagd=2 if freq2d!=. & freq3d==.  

gen iflag5 = (flagd == 5)
gen iflag4 = (flagd >= 4)

* Collapse to gvkey/date by computing weighted averages first of firm-years then normal averages of firms
collapse (mean) freq dur (sd) sd_freq=freq sd_dur=dur (median) med_flagd=flagd (min) min_flagd=flagd (mean) shareflag5 = iflag5 shareflag4 = iflag4 (count) seg_count=freq [aw=sales], by(gvkey year)

drop if seg_count < 2 
keep if year>=1995 & year<=2017

collapse (mean) freq dur shareflag5 shareflag4 (mean) sd_freq sd_dur (min) minmin_flagd = min_flagd (median) medmin_flagd = min_flagd (max) maxmin_flagd = min_flagd (median) med_seg_count = seg_count (min) min_seg_count =  seg_count, by(gvkey)

keep if shareflag5 > 0.5

rename gvkey id 
save ${dirTemp}/firm_rigidity, replace
